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CONSTRAINED OPTIMAL CONTROLLER FOR LINEAR SYSTEMS WITH 
STATE- AND CONTROL-DEPENDENT DISTURBANCE 

S. Basuthakur* 

Goddard Space Flight Center 


INTRODUCTION 

This study considers the mathematical problem of designing a constrained optimal, 
dynamic controller for linear systems with parameter uncertainty. Usually exact values 
of the system parameters are not known to the designer, and any modeling error is 
usually taken into account as additive white noise disturbance in the input channel, 
assuming that the correlation time of the noise process is much smaller than that of the 
controlled system. In the case of wide deviations of parameters, the intensities of the 
disturbances vary with the deviations of state and control matrices. Thus the previous 
assumption, that the correlation time of the noise process is negligible compared to that 
of the controlled system, is no longer valid. 

McLane (1) has shown that control-dependent noise occurs in modeling the thrust mis- 
alignment in a gas-jet thrusting system for the attitude control of a satellite. The state- 
dependent noise occurs in the momentum exchange method for regulating the angular 
precession of rotating spacecraft. Krasovskii (2), Wonham (3), McLane (4), and others 
1(1), (5), (6), and (7)] have proposed a stochastic formulation of the above problem, 
assuming that the intensities of the state- and control-dependent noise are known and 
have determined the time-varying feedback gain by minimizing a quadratic criterion. 

In this study, an alternative approach to the above problem is described. This approach 
assumes that nominal values of system matrices are available, although their precise values 
or variations are unknown, and it prescribes a minimax design for a time-invariant dynamic 
controller. The technique presented in this paper considers the uncertain parameters and 
their deviations to be an adversary wliich seeks to maximize a performance criterion, a 
criterion which the controller attempts to minimize. This minimaximization procedure 
leads to a conservative design, although it is argued that the performance criterion reflects 
a meaningful physical situation. Note that the effect of uncertain parameters, the stuie- 
and control-dependent noise, has a destabilizing influence on the control system; that is, 
it tends to maximize the performance criterion (4). 
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It is to be noted here that the system need not be governed by tto stochastic differen- 
tial equations since uncertain parameters or parameter variations from nominal are con- 
sidered in this paper as essentially nonrandom (8). In addition, the problem is posed with 
two additional constraints; first, that only the noise-corrupted outputs, not all the states, 
are available; and second, that the feedback gains (controller gains) are time-invariant. 

The second constraint seems more practical from the viewpoint of implementing the 
controller. Since it has been observed in references (3) and (5) that linearly state- and 
control-dependent noise of sufficient intensities may make stabilization by linear full- 
state feedback impossible (let alone the output feedback), even if the nominal system 
matrix pair (A 0 , B 0 ) is controllable in the usual sense, it is believed that a dynamic con- 
troller which uses noise-corrupted outputs has a better chance of stabilizing the system. 

Although the estimation of state from limited measurements may be provided by a 
Kalman filter, the problem becomes difficult in this case, due to the presence of parameter 
uncertainty. In addition, the dimension of the filter may be unnecessarily large. This 
subject has been the motivation of a number of papers in deterministic optimal design 
[(9), (10), and (11)]. Minimax controller designs have been proposed by many authors 
[(12), (13), and (14)] . However, most of these papers allow the control or its opponent 
to employ full-state feedback. 

The problem is treated here by minimizing, with respect to feedback matrices, and maxi- 
mizing, with respect to parameter uncertainty, a quadratic performance index which 
involves state, control, and parameter deviations. The necessary conditions which result 
are a set of nonlinear two point boundary value problems (TPBVP). The present formu- 
lation also permits the inclusion of situations where the passband of disturbances is not 
necessarily large compared to that of the controlled system. This approach is a generali- 
zation of the basic concepts advanced by Wonham (3), McLane (4), and others, in that 
the dynamic controller, rather than the output feedback, has a better chance of stabilizing 
the system and the formulation permits inclusion of colored state- and control-dependent 
noise. 

SYSTEM DESCRIPTION AND PROBLEM FORMULATION 

The system to be controlled is a linear system with state vector x_(t), output vector y(t), 


control vector u(t), and compensator state vector z_(t) related by 

x = A q 2L + B q u + (A - A 0 )_x + (B - B 0 )u + Dw(t), Plant state equation (1) 

z_ - Fj^+Gy, Compensator equation (2) 

y - Cx_ + (t), Measurement equation (3) 

u = Hz + Ny, Control equation. (4) 
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A 0 and B Q are nominal representations of matrices A and B, respectively, which may 
contain certain elements. Vectors x, z , y, ,u , and w have dimension n, p, q, r, and d, 
respectively. All matrices are time-invariant and have compatible dimensions; v (t) and 
w(t) are sample functions of zero mean, stationary, white-noise processes satisfying 


E [w(tj)w T (t 2 )] = W 5 (tj - t 2 ), E [v (tj)y T (t 2 )] = V 5 (t A - t 2 ), 
E [w.(tj)y_ T (t 2 )] = U T 5 (tj - t 2 ). 


( 5 ) 


Furthermore, the initial state of the plant is assumed to be random with a known mean 
and convariance. Having specified the dimension of the compensator, the designer is 
free to choose the matrices F, G, H, and N and the initial state z 0 . 


The system equations (1) through (4) may be rewritten so that F, G, H, and N appear as 
elements of a single matrix. For this purpose the following real vectors and matrices are 
defined with the dimensions of partitions indicated: 
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E [m (0)m (0) T ] , 


■*..», -[o' o|„-Mo «],■ (6) 

where I p and I q are identity matrices of dimension p and q respectively. Equations 
(1 ) through (4.) and (6) combine to give 

a AAA a AAA A AAA AAA, 

m = ($ 0 + B 0 NC)m + (D + B 0 NI)s + (W, + W 2 NC)m + W 2 N I s . (7) 


A 

The basic problem now reduces to select the feedback matrix N, in order to provide 

A A 

acceptable performance despite the presence of parameter uncertainties W x and . 

In order to determine N, a quadratic performance criterion involving state vector m , 

A' A 

control vector u, and parameter uncertainty W , W. is defined. This criterion is then 
maximized with respect to N. The uncertain parameters are therefore permitted to 
assume their worst values, limited only by penalties included in the criterion. Although 
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this type of minimax design is often conservative, it has the advantage of requiring 
relatively meager information about uncertain parameters. It should also be noted that 
the colored state- and control-dependent noise can be modeled by using additional states 
in the plant and suitably modifying matrix C. The dimension of z need not be modified, 
however, and even in the full state estimation problem it may be retained as the original 
x dimension. 

It should be noted that W, and W, will normally contain many zero entries. In order to 
restrict variation of W t and W 2 to the nonzero terms, it is helpful to decompose Wj and 
W 2 in a simple way. If a matrix W has n rows and m columns, it is always possible to 
express W as 



n iGiCi 


( 8 ) 


where matrices G. contain the nonzero terms and only the nonzero terms of W, D., and 
C. are known matrices, C is the minimum numbers of rows and columns, in any com- 
bination, which are required to cover the nonzero terms of W. Clearly, £ < min (n, m). 
Two examples illustrate this expansion. 


Example 1. nth order system with 1st order compensator: 
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Wj may assume this form whenever matrix A is in companion form and a first order 
compensator is used. 


A single term of equation (8) is adequate here since W has but one nonzero row. 


Example 2. 
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In this case, nonzero elements of W are covered by one column and one row, and hence 
the expansion has two terms. Care has been taken to insure that W 3 is included only 
once. By proper scaling with C. for a row term or Dj for a column term of equation (8), 
the range of parameter variations expected for all elements of G. can be made the same. 
This is useful in properly penalizing parameter variations in the performance criterion. 

It should be noted that the construction suggested assures that Cl has maximum rank. 


It is assumed, for simplicity in the developments which follow, that 

Wj = D t G, C,,V D a G 2 C 2 > 


(9) 


that is, Wj and W 2 are assumed to contain a single nonzero row or column. However, the 
procedure employed to obtain necessary conditions for a minimax solution can also be 
used in a straightforward manner when W t and W 2 require the more general expansion of 
equation (8). If equation (9) is substituted in equation (7), the result is a state equation 
in terms of feedback matrix N and “uncertainty” matrices Gj and G 2 , 
m = (& 0 +fc 0 NC)m + (D + B 0 NI)S. + (Dj G t C, + D 2 G 2 C 2 NC)m + D 2 G 2 C 2 NIs 


= <k 


AAA 

o + g 0 NC 


+ D. 


Gj C, 


A A 


A AAA 


D, G„ C, N C) m + (D + B 0 N I + D 2 G 2 C 2 N ^ . 


(10) 


Matrix A will be determined using the performance criterion 


J (A, Gj , G 2 , z^ - 


■HI 


G xz 
G zx Qz 


] [ i ] + - T RMdt 'i Tr ( K , c , G ; tK , G 2 c i] 


1 f‘ f f T f a t a t aaa] „ |C T N T RN ! | ■ 
— El |m T [q + C t N t RNC] m +2mT| HTRN ]y 


+ i v t N t RNv |dt -i.Tr [Kj Gj Gj + Kj G 2 G 2 ] 


(ID 


R, Kj and K 2 are positive definite, symmetric matrices. Q is symmetric and positive 
semidefinite; R and Q are constant. Tr in equation (11) denotes the trace. The trace 
terms have been introduced to limit the variation of the uncertain parameters G x and G 2 . 
Kj and K 2 may provide some flexibility in weighing elements of Gj and G 2 , but this is 
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best accomplished by proper sealing with D, , C t , D 2 , and C 2 as previously suggested. 
Frequently, K x and K 2 will merely be scalars. The minus sign is necessary in equation (11) 
because the criterion is to be maximized with respect to G t and G 2 . Initial time t 0 = 0 
and final time t f = «> are of primary interest. It must be assumed, therefore, that the 
dimension p of the compensator is adequate to stabilize the system so that the integral 
component of equation (11) remains finite as t f -► «>. 

It should be noted 

~ e[v t N t RNv ] = Tr [n t RNE(v v t )] = ^ Tr [n t RNV ] 5(0), 

5(0) is the impulse fulsion. This makes an unbounded contribution to J unless 
Tr[N T RNV] = 0. We shall assume that R and V are either positive definite or zero. 

Hence it is dear that either 

(a) R = 0, (b) V = 0, v (t) = 0, or (c) N = 0 


is required for a finite L. This result is not surprising. When the output is corrupted with 
white noise, design via quadratic cost involving u T Ru will lead to zero output feedback 
gain. What it really means is that the noise must be filtered before it is fed back to the 
system. Of course, nonzero output feedback gain can be obtained by setting R = 0, 
meaning J is independent of U or the noise-free measurement. In any case described 
above, equation (11) may be replaced by 


J(N,G i; G 2 ,z (t 0 )) 



+ Tr !> T A 


| m T [Q + C t N t RNC] 
(k,G,G[ t 


m 


K, G, GJ 


Addition of Tr $ T N] merely appends the constraint N - 0 when R =£ 0 and V ^ 0. 
Otherwise 0 = 0, and N = 0. 
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MINIMAXIMIZATION OF THE FINITE INTERVAL CRITERION, J 

Conditions to be satisfied by the matrix N, G, , G 2 and initial state z_ 0 which minimaxi- 
mize J will be obtained using an approach presented in references (15) and (16). 


Let 


J 


A A 

E[J], J = 



Ldt. 


(13) 
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= o, (14) 


A 

If N, Gj , G 2 and z_ 0 are to be extremum, then 



where the interchange of expectation and differentiation is assumed to be valid (17). The 
partial derivatives in equation (14) may be evaluated using the following well-known 
results ( 1 8). 

Lemma: 

If J = J(x(t 0 )) = 


f 


L(x,t)dt + P(x,t f ), 


where 


x. = f (x , t) , 


then 

where 



dJ 

(t 0 ) 


X (t 0 ) , 


3 

3x 


T dP (x (tf), t f ) 

(L + X T f), A(t f ) = — ' 


This lemma follows from the variational calculus where the first variation of J with respect 
to x(t 0 ) is X T (t 0 ) 5x(t 0 ). 

In order to apply the lemma here, elements of N, G t and G 2 are treated as additional 
states which satisfy 



= 0, g 2 = 0. 


(15) 


Vector multiplier X m will be used for the regular state constraint of equation ( 1 0), and 
matrix multiplier A N (t), A G (t), and A G2 (t) will be used for constraints of equation (15). 
Note that the Hamiltonian tf in the lemma will be independent of A N , A Gi , A Gz due to 
equation (15). Thus the Hamiltonian is 


H = [q + £ t N t RNc] m + X* [^ 0 + B 0 Nc)m + (6+B 0 NI ) s 

+ (Dj G l C 1 + D 2 G 2 C 2 ft C ) m + D 2 G 2 C 2 Nts_] + Tr [ £ T ft ] 

= Tr U- Qmni T j + Tr ( m X* + (d + B q NI+D 2 G 2 C 2 ft I ) s_ X* ] (16) 


+ Tr 
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where 
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( 21 ) 


* / A r A-t- A A A \ 

Q = (Q + C r N 1 RNC j (22) 

The gradient matrix expressions in equations (18), (19), and (20) have been obtained using 
the rules in reference (19). 

The lemma, the necessary conditions of equation (14), and integrated forms of equations 
(18), (19), and (20) combine to give 


9J 

8N 


= e [ft N (t 0 )| = e r [ 
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(24) 
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dJ 

dG 


2 J 


= E [a g (t 0 )] ---kJ' d;*„ (^ewr) 


N 1 


Cjdt - 


k 2 g 2 = o 


(25) 


Equations (23), (24), and (25) can be rearranged to yield simultaneous expressions for 
N, Gj, and G 2 which involve integrals of E [m m T 1 and E [X m m T ] . In order to evaluate 
these integrals, assume that 

A m (t) = K(t)m(t) + T?(t) , (26) 

satisfies equation (17) for some choice of K(t) and a(t). Substitutions of equations (26) 
and (10) in equation (17) indicate that the assumption is valid provided 

* * * 

K + KA 0 + aJk + Q = 0, K(t f ) = 0, (27) 

and 

$ 

n + a£ ti + K (6 + B 0 N I + D. 2 G 2 C 2 N't) s_ = 0, i? (t f ) = 0. (28) 

By expressing the solutions of equations (10) and (28) using the transition matrix asso- 
ciated with A 0 , it is straightforward to show 

E [m (t) ?i T (t)] = 0, 

E O (t) l t (t) ] = K(t) E [m (t)L T (t)] = -jK(t) (d + B Q N I + D 2 G 2 C 2 N I ) S , 


E [X ra (t)m T (t)] = K (t) P (t), 

r 'r y 1 / A A AA A a \ 

E [A m (t)s T (0] = K (t) (d + B 0 NI + D 2 G 2 C 2 Nl)s, 


where 


(29) 


(30) 


P (t) A E [m (t) m T (t)] 

is governed by 

* * T / A A AA Aa\ / A A AA AA \ T 

P = A 0 P + PA T 0 + ( D + B 0 N I + D 2 G 2 C 2 N I j S ^ D + B 0 N I + D 2 G 2 C 2 N I j , (31) 


and 


P(0) = E [m (0) m T (0)] = M 0 . 


(32) 
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Hence, equations (23), (24), and (25) become 




When z 0 is assigned arbitrarily, N is determined by simultaneous solution of equations (27) 
and (31) through (35). As t f -> 00 in equations (33), (34), and (35), the integrals are 
dominated by steady-state values of K and P. As t f -*■ °o, however, J will not in general 
remain finite. 


If z 0 is to be chosen optimally, an expression for E [\ (0)} will be required. From 
equations (26), (28), and (6) it is clear that 



+ E [t?(0)] = K (0) 



(36) 


If matrices K and P are partitioned according to 



it is clear from equations (14), (37), and the lemma that 



E [M°)] = ^\(0)z_ 0 = 0, 


or z 0 = - K; 1 (0) (0) x 0 . (38) 

It will be assumed that an inverse of K z exists. Simultaneous solution of equations (27), 
(31) through (35), and (38) yields the optimal N, z_ 0 . 
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CALCULATION OF N, G, , AND G 2 

It is clear from above that the optimal solution will involve solving a nonlinear two point 
boundary value problem (TPBVP). Numerous numerical methods have been developed to 
solve TPBVP (18), and the matter will not be discussed here except to point out two 
straightforward algorithms for which there is no assurance of convergence. 

Method 1: 

V 

A 

a. Specify an N, G^and G 2 , which if possible stabilizes the system (usually not a 
small task); 

b. Integrate equation (27) to obtain K(t) on [0, t f ] ; 

c. Integrate equation (31) using equation (38) for boundary information to obtain 
P(t) on [0, t f ] ; 

d. Perform integrations required in equation (33); . 

A 

e. Solve equation (33) for a new N; 

f. Return to steps b and c to update K and P; 

g. Revise G t and G 2 according to equations (34) and (35) and repeat the entire 
process until the convergence is achieved. 

Method 2: 

A 

a. Proceed as in equation (3) except that in updating N, K, and P use equation (31) 
to obtain K ; 

b. Solve equations (27) and (33) simultaneously for N and P. 

DISCUSSION AND CONCLUSION 

Two special cases of equations (31) through (35) are of interest. In the first case, when 
Kj and K 2 are infinite, thereby permitting no parameter variation, matrices G : and G 2 
go to zero. For output feedback (P = 0) only, equations (27), (31), and (33) reduce 
respectively to 

k + (A + B N C) T K + K(A + B N C) + Q + C T N T R N C = 0 K (t f ) - 0, 

P = (A+BNQP + P (A + B N C) + DWD T , P(0) = M 0 , 



11 



These equations are comparable to those obtained by McLane (4). McLane, however, 
assumes that N = N(t) which, as a result, yields a modified form of equation (39). 

The second special case occurs when the system is noise free; that is, S = 0 and j) = 0. 
Equation (31) becomes homogeneous and equation (33) assumes the same form as 
equation (39) with matrices R, N, C, and B “hatted.” As t f -*•<» it can be shown that 
in absence of noise and parameter uncertainty, 

M. A I P (t) dt 

•^o 

satisfies 

$ + $NC)M + M(ft + §NC) + M 0 = 0, (40) 

while equations (33) and (27) reduce respectively to 

RNC m6 t + t T Km6 t = 0, (41) 

and 

(i + BNC) T K + K (k + B N C) + Q + 6 T N T ft N C = 0 . (42) 

The conditions of equations (40), (41), and (42), except for variations due to a difference 
in criteria, are basically those discussed in reference (8). The design of a constant con- 
strained optimal controller for linear systems subjected to stationary random inputs, 
state-dependent, control-dependent, and measurement noise, has been described. 

It has been shown that the optimal controller gains may be determined by solving a non- 
linear matrix TPBVP. The extent to which this approach will be applied in practice 
depends very much on the case with which computational problems can be solved. This 
is true of other approaches presented in references (2), (3), (4) and (9), (10), ( 11). The 
present approach is an alternative formulation of the problem of controlling a system with 
state-dependent and control-dependent noise. It is also a generalization of the problem 
of references (3) and (4), in the sense that no assumptions regarding intensities of state- 
dependent and control-dependent noise or of the availability of complete state feedback 
have been made, and that certain dynamics in the controller have been added to generate 
an estimate of feedback using only available outputs. 

It should be pointed out that certain questions regarding existence and uniqueness of the 
solutions certainly exist (7). In addition, it has been assumed that the controller of 
dimension p is sufficient to stabilize the system. Uniqueness of the optimal compensator 
is not expected. For example, if y is scalar, G is arbitrary; then H can adjust for any value 
of G. 

Goddard Space Flight Center 

National Aeronautics and Space Administration 
Greenbelt, Maryland February 13, 1974 
630*21-75-01-51 
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